Unfortunately,the per-iteration cost of maintaining this adaptivedistribution for gradient estimation is more than calculating the full gradient itself, which we call the chicken-and-the-egg loop. As a result, the false impression of faster convergence in iterations, inreality,leads to slower convergence in time.

Deep neural networks have achieved impressive performance in many areas. Designing a fast and provable method for training neural networks is a fundamental question in machine learning. The classical training method requires paying Ω(mnd) cost for both forward computation and backward computation, where m is the width of the neural network, and we are given n training points in d-dimensional space.

The emergence of the Internet-of-Things (IoT) sheds light on applying the machine teaching (MT) algorithms for online personalized education on home devices.

These authors contributed equally to this work.

Section 4.2 demonstrates addressthechallenges (Figure 1 contains of Scatterbrain).

IntheDistanceOracle problem,thegoalistopreprocess nvectorsx1,x2,...,xn in a d-dimensional metric space(Xd, l) into a cheap data structure, so that given a query vectorq Xd and a subsetS [n] of the input data points, all distances q xi l forxi S canbequicklyapproximated(fasterthanthetrivial d|S|querytime).